Digitization and Search, A Non-Traditional Use of HPC

We describe our efforts in developing an open source cyberinfrastructure to provide a form of automated search of handwritten content within large digitized document archives. Such collections are a treasure trove of data ranging from decades ago to as far as the present. The information contained in these collections is also very relevant to both researchers who might extract numerical or statistical data from such sources as well as the general public. With the push to digitize our paper archives we are, however , faced with the fact that though these digital versions are easier to share, they are not trivially searchable as the digitiza-tion process produces image data and not text. This inability to find and/or identify contents within these collections makes this data largely unusable without a lengthy and costly manual transcription process carried out by human beings. To carry out the search we build on top of a computer vision technique called word spotting. A form of content based image retrieval, it avoids the still difficult task of directly recognizing the text by allowing a user to search using a query image containing handwritten text and ranking a database of images in terms of those that contain more similar looking content. In order to make this search capability available on a large archive, three computationally expensive pre-processing steps are required, Figure 1. First, forms are segmented into individual units of handwritten information. In the case of the 1930 Census data collection, which contains approximately 3.6 million spreadsheet-like forms, this entails breaking the form images into sub-images of individual cells that contain the information about the individuals recorded in the Census. Second, the extracted sub-images are processed so as to extract features and descriptors that represent the handwritten contents within them. The utilized word spotting method results in a 30 dimensional vector derived from the frequency components of the darker ink pixels [1]. The distance between two such signature vectors can be used to determine how similar the handwritten contents of their cell sub-images are. Third, an indexing step organizes these extracted signatures into a binary tree structure to enable fast user queries. For the 1930 Census data this involves organizing nearly 7 billion sub-images using a hierarchical agglomerative clustering. Organizing the entire collection at once isn't practical, thus we instead break this step into multiple index construction steps based on states, categories, and microfilm reels passing the …